{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "148ba633-7e61-4423-b7b9-37cb0d02bc6b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "0ee74c9d-22c9-4317-a8f6-af85ebd2394c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import Ridge\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54da8967-8678-458b-af95-46b292551ae0",
   "metadata": {},
   "source": [
    "# 岭回归模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c7483b2-4de1-4c38-8f15-e731e875100d",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## 一、岭回归的原理"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a8fd0cc-80c6-4461-8638-520fc5c4337b",
   "metadata": {},
   "source": [
    "缩减系数来理解数据\n",
    "\n",
    "如果数据的特征比样本点还多应该怎么办？是否还可以使用线性回归和之前的方法来做预测？\n",
    "\n",
    "答案是否定的，不能使用前面介绍线性回归的方法。这是因为输入数据的矩阵X不是满秩矩阵。\n",
    "\n",
    "非满秩矩阵在求逆矩阵的时候会出现问题。\n",
    "\n",
    "为了解决这个问题，统计学家引入了岭回归（ridge regression）的概念"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daee0872-294b-4869-84cb-8e20f5c98235",
   "metadata": {},
   "source": [
    "简单来说，岭回归就是在矩阵$X^TX$上加上一个$\\lambda E$从而使得矩阵非奇异，进而能对$X^TX+\\lambda E$求逆，其中矩阵E是一个mxm的单位矩阵。对角线上元素全为1，其他元素全0。而$\\lambda$是一个用户定义的数值，后面会做介绍。在这种情况下，回归系数的计算公式将变成：\n",
    "    \n",
    "$$\\hat w = (X^TX+\\lambda E)X^Ty$$\n",
    "\n",
    "岭回归最先用来处理特征数多于样本数的情况，现在也用于在估计中加入偏差，从而得到更好的估计。这里通过引入$\\lambda$来限制了所有的w之和，通过引入该惩罚项，能够减少不重要的参数，这个技术在统计学中也叫缩减（shrinkage）"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5d8f859-3248-452b-8bba-ba8361849e9d",
   "metadata": {},
   "source": [
    "缩减方法可以去掉不重要的参数，因此能更好地理解数据。此外，与简单的线性回归相比，缩减法能够取得更好地的预测效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e40ddab0-5639-45d5-8fff-c891c1ed0003",
   "metadata": {},
   "source": [
    "岭回归是加了二阶正则项（$\\lambda E$）的最小二乘法，主要适用于过拟合严重或各变量之间存在多重共线性的时候，岭回归是有偏差bias的，这样的bias是为了让方差variance更小"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2dfa2da8-bccb-43e2-9b34-68169fad640a",
   "metadata": {},
   "source": [
    "1.岭回归可以解决特征数量比样本量多的问题\n",
    "\n",
    "2.岭回归作为一种缩减算法可以判断哪些特征重要或者不重要，有点类似于降维的效果\n",
    "\n",
    "3.缩减算法可以看作是对一个模型增加偏差的同时减少方差\n",
    "\n",
    "岭回归用于处理下面两类问题：\n",
    "\n",
    "1.数据点少于变量个数\n",
    "\n",
    "2.变量间存在共线性（最小二乘回归得到的系数不稳定，方差很大）"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01b6bee0-eee9-45c6-ba66-0a1282068e9e",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## 二、举例说明"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adafc1ae-9ec2-45c1-b7c2-73388748484f",
   "metadata": {},
   "source": [
    "以50个方程，200个未知数为例，方程组有无穷解。求不出唯一解。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7d097564-b2ff-4256-a7ba-3af011ac3bbd",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.randn(50, 200)\n",
    "y = np.random.randn(50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e865ea4d-1dad-4dcf-8f6b-755e356c71fe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.13556708,  0.25554329, -1.16770836, ..., -0.70509332,\n",
       "        -2.01717215, -1.48933474],\n",
       "       [-1.18314699,  0.81302632, -0.02154716, ..., -0.08791472,\n",
       "        -0.63359608,  0.2425059 ],\n",
       "       [-1.33206395,  1.11361876, -1.21634781, ..., -1.15005896,\n",
       "         0.35689718, -0.91250941],\n",
       "       ...,\n",
       "       [ 1.1458266 ,  0.6703288 , -1.22888246, ...,  0.75284994,\n",
       "         1.05150321,  0.91074125],\n",
       "       [ 0.07659401, -1.19516272, -0.62864463, ...,  0.33332068,\n",
       "         0.30601   ,  0.68648358],\n",
       "       [ 1.80791321,  2.16132577,  0.8513482 , ..., -0.32506386,\n",
       "        -0.28443066,  0.99040965]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ec4b8cde-96da-4ccf-a34e-bc07b7452b9f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.03986432, -0.11304709,  0.15611716,  0.42626324, -0.59570809,\n",
       "       -0.85128164,  0.57054041, -0.67267058,  0.3634413 , -0.68930036,\n",
       "       -0.84381909,  0.42812528, -0.06631721,  0.5412897 ,  0.88111843,\n",
       "        2.04246859, -0.96326735, -1.48475211,  1.01902741, -0.61929921,\n",
       "       -1.20966424, -1.08841864,  0.07796065, -0.60380565, -0.19014749,\n",
       "        1.24030829, -0.76311825,  1.38970606, -0.06471864,  1.02808702,\n",
       "       -1.62119676, -0.62907233, -0.6757333 , -0.04146303, -1.29655333,\n",
       "       -0.47352394,  0.00731335, -0.9779559 , -0.47531097,  0.62222724,\n",
       "        0.04560572, -0.65081021, -1.4916066 ,  0.81822799,  0.79567237,\n",
       "        0.47851657,  0.81332116, -1.07917033,  1.38992608,  1.9298173 ])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3ab7749-a16d-449f-b3d2-63335d5b5cae",
   "metadata": {},
   "source": [
    "### 1、使用普通线性回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "1334a4ec-b76c-4417-9082-2979a2c32a0b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearRegression</label><div class=\"sk-toggleable__content\"><pre>LinearRegression()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "LinearRegression()"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "linear = LinearRegression()\n",
    "linear.fit(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "21b7a995-6fb5-44a4-bf04-39caa2210f1f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 这种情况时，评估指标 R^2=1 表完美拟合。\n",
    "# 这时候的完美拟合是不正确的。\n",
    "linear.score(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7327dfe5-7cec-4e13-a5e5-fe9e189b2adc",
   "metadata": {},
   "source": [
    "正规方程：\n",
    "$$\n",
    "\\theta = (X^TX)^{-1}X^Ty\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "6f47b7bb-c811-452c-adcf-88b7da1b43c4",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 在使用正规方程的时候，为了让 y = w0x0 + w1x + w2x 中的w0能被算出来，所以给x0赋个1\n",
    "X = np.hstack((np.ones(shape=(50, 1)), x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "9757ece3-23de-4cfb-a59b-2508def06d7c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.        ,  1.13556708,  0.25554329, ..., -0.70509332,\n",
       "        -2.01717215, -1.48933474],\n",
       "       [ 1.        , -1.18314699,  0.81302632, ..., -0.08791472,\n",
       "        -0.63359608,  0.2425059 ],\n",
       "       [ 1.        , -1.33206395,  1.11361876, ..., -1.15005896,\n",
       "         0.35689718, -0.91250941],\n",
       "       ...,\n",
       "       [ 1.        ,  1.1458266 ,  0.6703288 , ...,  0.75284994,\n",
       "         1.05150321,  0.91074125],\n",
       "       [ 1.        ,  0.07659401, -1.19516272, ...,  0.33332068,\n",
       "         0.30601   ,  0.68648358],\n",
       "       [ 1.        ,  1.80791321,  2.16132577, ..., -0.32506386,\n",
       "        -0.28443066,  0.99040965]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "bb45dedb-7e55-4d73-bfc5-486960bb2e61",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2.79925318e+13, -4.84103916e+12, -7.05962835e+13, ...,\n",
       "        -3.57926907e+13, -1.02409732e+13, -1.71636803e+13],\n",
       "       [ 4.46745127e+13,  4.47078277e+12,  8.80669799e+12, ...,\n",
       "         6.13634258e+12,  4.94361266e+13, -1.31487609e+13],\n",
       "       [ 4.53020519e+13,  4.69466698e+13,  8.59319349e+13, ...,\n",
       "         4.12881888e+13,  1.05117485e+14,  5.14876301e+13],\n",
       "       ...,\n",
       "       [-1.11684181e+13,  9.84495166e+11, -1.46963954e+13, ...,\n",
       "         1.27203755e+13, -2.08371159e+12, -9.94328391e+12],\n",
       "       [ 7.09080545e+13,  1.53953459e+13, -1.59003615e+13, ...,\n",
       "        -1.39677294e+13,  6.95964231e+13,  2.95844645e+13],\n",
       "       [-3.31766111e+13, -1.78005135e+13, -7.35447708e+13, ...,\n",
       "        -2.52581314e+13, -7.53434405e+13, -5.68587540e+13]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# X^TX是一个奇异矩阵\n",
    "np.linalg.inv(X.T @ X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "e6263351-6e4b-4c45-a0e5-ebc90bf1bf0c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-9.35050698e+02, -3.19360698e+02, -9.91169338e+02, -3.09202780e+03,\n",
       "        1.47055093e+03, -8.10588846e+02, -1.99120532e+02, -1.26677296e+03,\n",
       "        3.45079204e+02,  1.10629280e+02, -3.07409427e+02, -1.51979120e+03,\n",
       "        8.23728775e+02, -1.78248338e+03,  1.22610979e+03, -6.56206663e+01,\n",
       "        1.38909626e+03, -2.18444689e+02,  8.38457014e+02,  6.96589764e+02,\n",
       "        1.43718518e+03,  1.94831009e+03, -7.61749890e+02, -1.06386605e+02,\n",
       "        2.19751820e+03,  8.78296540e+02, -2.13878488e+03, -8.09816224e+02,\n",
       "        5.72789653e+02,  3.09340348e+02, -1.55420692e+03, -7.65743646e+02,\n",
       "        2.60707635e+02,  7.31066148e+02, -7.65403499e+02,  6.75179014e+02,\n",
       "       -1.46280204e+03,  6.71177108e+02,  1.04711828e+03,  4.28751812e+02,\n",
       "        1.33650655e+03,  1.28326866e+03,  4.42258026e+02,  6.05819270e+02,\n",
       "        7.30214973e+02,  6.87973491e+02,  2.22084285e+03, -1.62549289e+03,\n",
       "        1.24907902e+03, -2.64542496e+01, -1.97168016e+01,  3.33994725e+00,\n",
       "        4.81213529e+00, -4.46330474e+00, -2.53759809e+01,  8.75449876e+00,\n",
       "        3.07125997e+01, -1.32275659e+01,  8.73704424e+00, -2.11600829e+01,\n",
       "       -3.66494356e+00,  8.56584436e+00,  2.40106955e+00,  1.15368256e+01,\n",
       "       -1.98149757e+01, -9.43374090e+00, -1.95517808e+01,  6.68689128e+00,\n",
       "        1.64012113e+00,  6.20068097e+00,  6.42120309e+00,  8.78014425e+00,\n",
       "       -3.75568242e+00,  1.48763392e+00, -1.03749493e+01, -7.15585380e+00,\n",
       "       -3.90918231e+01,  7.76048499e+00,  1.69161312e+01,  2.03505233e+00,\n",
       "        9.51683105e+00,  9.22760702e+00, -3.19072598e+00,  3.23242531e+00,\n",
       "        1.13945157e+01,  1.70111666e+01,  1.04790495e+01, -6.70385582e+00,\n",
       "        1.59785225e+01, -2.23102478e+01, -7.18208489e+00, -1.61654056e+01,\n",
       "        1.46325690e+01, -2.12052665e+01,  1.41753479e+01, -9.45642235e+00,\n",
       "        8.93672297e+00, -1.23727896e+01,  3.66693322e+00, -7.21793833e+00,\n",
       "       -1.40821781e+01,  5.63117024e-01, -7.68384597e+00,  2.40927591e+00,\n",
       "        3.98095343e+00,  3.19897668e+00, -5.87127424e+00, -7.52766754e+00,\n",
       "        3.50399117e+00,  1.09153219e+01,  7.99032250e-01, -1.48273403e+01,\n",
       "       -2.18839282e+00, -1.21906446e+01,  1.01121905e+01,  1.17896792e+01,\n",
       "       -7.28104220e+00,  1.05510755e+01,  5.34140605e+00,  5.26830966e+00,\n",
       "       -5.45167520e+00,  2.75636162e+00,  6.78697751e+00,  8.61385336e-01,\n",
       "        3.98715339e+00, -1.39546433e+00, -2.85587120e+00, -7.07646424e+00,\n",
       "       -8.27767345e-02, -4.97700187e+00, -6.60617002e+00, -2.64756234e-01,\n",
       "        8.25394937e+00,  4.01104789e+00, -6.79060460e-02,  1.34255346e+00,\n",
       "       -2.84008775e+00, -4.92816753e+00,  5.19024399e+00,  1.04232978e+01,\n",
       "        8.85656250e-01, -1.79663921e+00, -7.57014177e+00,  3.64210800e+00,\n",
       "        7.94370574e+00,  2.06611663e+01,  1.24847254e+01,  4.06899099e+00,\n",
       "        1.27299419e+00, -5.28871255e+00, -1.10248271e+00, -1.57006647e+00,\n",
       "        1.90817105e+00, -1.55284169e+00,  5.30554592e+00, -1.04989848e+01,\n",
       "        5.87725580e+00, -3.84681541e+00,  2.20820509e+00,  7.87881988e-01,\n",
       "        9.72107337e-01,  1.24198768e+00,  3.51905160e+00,  1.72918087e+00,\n",
       "       -5.60702995e+00,  3.05055615e+00, -4.22493592e-01,  1.85155001e+00,\n",
       "       -2.41037822e+00,  1.08034183e+00, -9.82484295e-01, -1.34904077e+00,\n",
       "        7.04003547e+00, -4.56944020e+00,  1.82054819e+00, -2.97589866e+00,\n",
       "       -1.55578968e+00,  4.54194633e+00,  1.98776211e+00, -1.06286085e+00,\n",
       "       -8.28927109e-02,  2.87491926e+00,  1.44948148e+00,  5.71963327e+00,\n",
       "        2.80358043e+00,  6.66889097e-01, -2.00639578e+00, -2.00113853e+00,\n",
       "        1.63521966e+00, -3.23789726e+00, -2.66141099e+00, -4.37029414e+00,\n",
       "        9.61894022e-01,  9.37815389e-01, -8.74523072e-02,  3.74320101e-01,\n",
       "        2.02205430e+00, -1.35751196e+00,  2.20714739e+00,  1.96254524e+00,\n",
       "       -2.32039509e+00])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 用正规方程算出来的结果没有意义\n",
    "np.dot(np.dot(np.linalg.inv(np.dot(X.T, X)), X.T), y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3df6ec3-2810-421a-b11a-9cd0dfdfa379",
   "metadata": {},
   "source": [
    "### 2、使用岭回归"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d698bf83-6175-4978-a023-6bd2c677eadb",
   "metadata": {},
   "source": [
    "Ridge(alpha=1)默认alpha岭参数为1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "578c4082-b9f3-496d-b04e-925cc6caca10",
   "metadata": {},
   "outputs": [],
   "source": [
    "ridge = Ridge()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "89241f23-47af-4ca9-9f69-7314574145f4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {color: black;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Ridge()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Ridge</label><div class=\"sk-toggleable__content\"><pre>Ridge()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "Ridge()"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge.fit(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "397b8dfc-b55b-446a-a017-979c1a772071",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9999297177246251"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge.score(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab39dc0a-2a5b-468e-848c-c646cf69c9b5",
   "metadata": {},
   "source": [
    "alpha岭参数越大，对数据的扰动越大"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "3052523c-1c71-4c12-b3ea-e96ae3c2859a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9999999928135135"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge = Ridge(alpha=0.01)\n",
    "ridge.fit(x, y)\n",
    "ridge.score(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "c8338b82-9422-4113-b7a8-da428e539bab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9941792860994355"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge = Ridge(alpha=10)\n",
    "ridge.fit(x, y)\n",
    "ridge.score(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e95cb4c2-db1c-4ce4-bd95-8dd6b2b2b221",
   "metadata": {},
   "source": [
    "岭回归优化的效果一般，能够缩减的系数比较有限，表现和普通的线性回归差不多"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5c2cc1c-f12a-40df-83ce-a0d170fc3a21",
   "metadata": {},
   "source": [
    "### 3、为什么加了λE就可以求出唯一解"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "baf9bd35-0491-449c-a7a8-8a7b8a869a07",
   "metadata": {},
   "source": [
    "探讨一下岭回归，为什么加了λE就可以求出唯一的w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "5ff96020-e056-42b8-92f0-6ef6ef929855",
   "metadata": {},
   "outputs": [],
   "source": [
    "n = np.array([\n",
    "    [1, 2, 3],\n",
    "    [2, 3, 5],\n",
    "    [2, 4, 6]\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "f55e1505-6073-4459-83fd-5884a9332ea5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2.81474977e+14,  2.81474977e+14, -2.81474977e+14],\n",
       "       [ 2.81474977e+14,  2.81474977e+14, -2.81474977e+14],\n",
       "       [-2.81474977e+14, -2.81474977e+14,  2.81474977e+14]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# X^TX是一个奇异矩阵，是求不出逆矩阵的，这里之所以能显示出结果是由于np的处理\n",
    "np.linalg.inv(np.dot(n.T, n))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "17ad657a-2a8f-412d-9624-8182d1cb0e16",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 查看矩阵的秩\n",
    "np.linalg.matrix_rank(n)  # 结果是矩阵 n 的秩为 2 不是一个满秩矩阵，n^Tn肯定没办法求逆"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10b28cda-b569-4b87-8e92-4ae74925dbf0",
   "metadata": {},
   "source": [
    "加入了λE看看会有什么变化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "21c8df22-4ca2-4d7a-b41b-6e3309338fc7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 2, 3],\n",
       "       [2, 3, 5],\n",
       "       [2, 4, 6]])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "4b5f2888-091d-4e40-8c16-33fad9a56920",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.01, 0.  , 0.  ],\n",
       "       [0.  , 0.01, 0.  ],\n",
       "       [0.  , 0.  , 0.01]])"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lambdaE = 0.01 * np.eye(3)\n",
    "lambdaE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8cb70f2-7c90-4936-8ee7-938fead7e42d",
   "metadata": {},
   "source": [
    "加入λE的正规方程：\n",
    "$$\n",
    "(X^TX+\\lambda E)^{-1}X^Ty\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "602b4327-f2f2-465c-a15e-4e5ed72beb3e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 相加之后，就变成了满秩矩阵，所以可以计算逆矩阵\n",
    "np.linalg.matrix_rank(n + lambdaE)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dc1931c-91da-4486-a938-63d131ee6af9",
   "metadata": {},
   "source": [
    "## 三、样本数比特征数多的时候，岭回归和线性回归效果一样"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3728c65-19d5-431f-a36a-dae7c76abbb7",
   "metadata": {},
   "source": [
    "用糖尿病数据集验证一下岭回归和线性回归的效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "70b3b788-19dd-4d64-ae73-8a0544afceb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_diabetes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "fe0b03ed-3549-4b6d-a453-abf4d310be2d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 在样本数比特征数多的时候，岭回归和线性回归的效果基本一样\n",
    "diabetes = load_diabetes()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "68e86257-4c87-49b3-8a0e-c05a95d4b044",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _diabetes_dataset:\n",
      "\n",
      "Diabetes dataset\n",
      "----------------\n",
      "\n",
      "Ten baseline variables, age, sex, body mass index, average blood\n",
      "pressure, and six blood serum measurements were obtained for each of n =\n",
      "442 diabetes patients, as well as the response of interest, a\n",
      "quantitative measure of disease progression one year after baseline.\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "  :Number of Instances: 442\n",
      "\n",
      "  :Number of Attributes: First 10 columns are numeric predictive values\n",
      "\n",
      "  :Target: Column 11 is a quantitative measure of disease progression one year after baseline\n",
      "\n",
      "  :Attribute Information:\n",
      "      - age     age in years\n",
      "      - sex\n",
      "      - bmi     body mass index\n",
      "      - bp      average blood pressure\n",
      "      - s1      tc, total serum cholesterol\n",
      "      - s2      ldl, low-density lipoproteins\n",
      "      - s3      hdl, high-density lipoproteins\n",
      "      - s4      tch, total cholesterol / HDL\n",
      "      - s5      ltg, possibly log of serum triglycerides level\n",
      "      - s6      glu, blood sugar level\n",
      "\n",
      "Note: Each of these 10 feature variables have been mean centered and scaled by the standard deviation times the square root of `n_samples` (i.e. the sum of squares of each column totals 1).\n",
      "\n",
      "Source URL:\n",
      "https://www4.stat.ncsu.edu/~boos/var.select/diabetes.html\n",
      "\n",
      "For more information see:\n",
      "Bradley Efron, Trevor Hastie, Iain Johnstone and Robert Tibshirani (2004) \"Least Angle Regression,\" Annals of Statistics (with discussion), 407-499.\n",
      "(https://web.stanford.edu/~hastie/Papers/LARS/LeastAngle_2002.pdf)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 查看数据集描述\n",
    "print(diabetes.DESCR)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27054b1b-4f75-4a9b-bfba-4597cd8a5d54",
   "metadata": {},
   "source": [
    "442个样本，10个特征。样本数比特征数要多（方程数量大于未知数个数）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "e5e4e824-5025-49b4-b4d6-9d512f19b9fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = diabetes.data\n",
    "target = diabetes.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "c8d7e1a3-dd84-4d47-87ac-c7d530145dd9",
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_names = diabetes.feature_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "d83a714a-15c1-4355-a2f9-039b7d52e186",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.03807591,  0.05068012,  0.06169621, ..., -0.00259226,\n",
       "         0.01990749, -0.01764613],\n",
       "       [-0.00188202, -0.04464164, -0.05147406, ..., -0.03949338,\n",
       "        -0.06833155, -0.09220405],\n",
       "       [ 0.08529891,  0.05068012,  0.04445121, ..., -0.00259226,\n",
       "         0.00286131, -0.02593034],\n",
       "       ...,\n",
       "       [ 0.04170844,  0.05068012, -0.01590626, ..., -0.01107952,\n",
       "        -0.04688253,  0.01549073],\n",
       "       [-0.04547248, -0.04464164,  0.03906215, ...,  0.02655962,\n",
       "         0.04452873, -0.02593034],\n",
       "       [-0.04547248, -0.04464164, -0.0730303 , ..., -0.03949338,\n",
       "        -0.00422151,  0.00306441]])"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 数据集的特征数据\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "0a579ec8-da6d-4b01-a1e4-a7481410f9e7",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([151.,  75., 141., 206., 135.,  97., 138.,  63., 110., 310., 101.,\n",
       "        69., 179., 185., 118., 171., 166., 144.,  97., 168.,  68.,  49.,\n",
       "        68., 245., 184., 202., 137.,  85., 131., 283., 129.,  59., 341.,\n",
       "        87.,  65., 102., 265., 276., 252.,  90., 100.,  55.,  61.,  92.,\n",
       "       259.,  53., 190., 142.,  75., 142., 155., 225.,  59., 104., 182.,\n",
       "       128.,  52.,  37., 170., 170.,  61., 144.,  52., 128.,  71., 163.,\n",
       "       150.,  97., 160., 178.,  48., 270., 202., 111.,  85.,  42., 170.,\n",
       "       200., 252., 113., 143.,  51.,  52., 210.,  65., 141.,  55., 134.,\n",
       "        42., 111.,  98., 164.,  48.,  96.,  90., 162., 150., 279.,  92.,\n",
       "        83., 128., 102., 302., 198.,  95.,  53., 134., 144., 232.,  81.,\n",
       "       104.,  59., 246., 297., 258., 229., 275., 281., 179., 200., 200.,\n",
       "       173., 180.,  84., 121., 161.,  99., 109., 115., 268., 274., 158.,\n",
       "       107.,  83., 103., 272.,  85., 280., 336., 281., 118., 317., 235.,\n",
       "        60., 174., 259., 178., 128.,  96., 126., 288.,  88., 292.,  71.,\n",
       "       197., 186.,  25.,  84.,  96., 195.,  53., 217., 172., 131., 214.,\n",
       "        59.,  70., 220., 268., 152.,  47.,  74., 295., 101., 151., 127.,\n",
       "       237., 225.,  81., 151., 107.,  64., 138., 185., 265., 101., 137.,\n",
       "       143., 141.,  79., 292., 178.,  91., 116.,  86., 122.,  72., 129.,\n",
       "       142.,  90., 158.,  39., 196., 222., 277.,  99., 196., 202., 155.,\n",
       "        77., 191.,  70.,  73.,  49.,  65., 263., 248., 296., 214., 185.,\n",
       "        78.,  93., 252., 150.,  77., 208.,  77., 108., 160.,  53., 220.,\n",
       "       154., 259.,  90., 246., 124.,  67.,  72., 257., 262., 275., 177.,\n",
       "        71.,  47., 187., 125.,  78.,  51., 258., 215., 303., 243.,  91.,\n",
       "       150., 310., 153., 346.,  63.,  89.,  50.,  39., 103., 308., 116.,\n",
       "       145.,  74.,  45., 115., 264.,  87., 202., 127., 182., 241.,  66.,\n",
       "        94., 283.,  64., 102., 200., 265.,  94., 230., 181., 156., 233.,\n",
       "        60., 219.,  80.,  68., 332., 248.,  84., 200.,  55.,  85.,  89.,\n",
       "        31., 129.,  83., 275.,  65., 198., 236., 253., 124.,  44., 172.,\n",
       "       114., 142., 109., 180., 144., 163., 147.,  97., 220., 190., 109.,\n",
       "       191., 122., 230., 242., 248., 249., 192., 131., 237.,  78., 135.,\n",
       "       244., 199., 270., 164.,  72.,  96., 306.,  91., 214.,  95., 216.,\n",
       "       263., 178., 113., 200., 139., 139.,  88., 148.,  88., 243.,  71.,\n",
       "        77., 109., 272.,  60.,  54., 221.,  90., 311., 281., 182., 321.,\n",
       "        58., 262., 206., 233., 242., 123., 167.,  63., 197.,  71., 168.,\n",
       "       140., 217., 121., 235., 245.,  40.,  52., 104., 132.,  88.,  69.,\n",
       "       219.,  72., 201., 110.,  51., 277.,  63., 118.,  69., 273., 258.,\n",
       "        43., 198., 242., 232., 175.,  93., 168., 275., 293., 281.,  72.,\n",
       "       140., 189., 181., 209., 136., 261., 113., 131., 174., 257.,  55.,\n",
       "        84.,  42., 146., 212., 233.,  91., 111., 152., 120.,  67., 310.,\n",
       "        94., 183.,  66., 173.,  72.,  49.,  64.,  48., 178., 104., 132.,\n",
       "       220.,  57.])"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 数据集的标签\n",
    "target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "05e2b364-8db8-4a29-909b-e5e33fca0047",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['age', 'sex', 'bmi', 'bp', 's1', 's2', 's3', 's4', 's5', 's6']"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 特征的名字\n",
    "feature_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "b058baeb-072e-49e2-a5da-7fa9f41c566c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 划分数据集\n",
    "X_train, X_test, y_train, y_test = train_test_split(data, target, test_size=42, random_state=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "cf8980f6-cc78-48d8-9d83-538e5925beb9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.2986943111235948"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "linear = LinearRegression()\n",
    "linear.fit(X_train, y_train)\n",
    "linear.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "6c7f20fb-3ac6-478d-be42-09a048587085",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.2980477814331671"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge = Ridge(alpha=0.001)\n",
    "ridge.fit(X_train, y_train)\n",
    "ridge.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd414ff2-33ca-4e2f-93a5-52bffd9b2e4d",
   "metadata": {},
   "source": [
    "线性回归R^2 = 0.2986943111235948\n",
    "\n",
    "岭回归R^2 = 0.2980477814331671\n",
    "\n",
    "两者效果很相近"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c190eb93-6944-41cc-a86c-7dc766589ca4",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
